Constructive Type-Logical Supertagging with Self-Attention Networks

We propose a novel application of self-attention networks towards grammar induction. We present an attention-based supertagger for a refined type-logical grammar, trained on constructing types inductively. In addition to achieving a high overall type accuracy, our model is able to learn the syntax of the grammar's type system along with its denotational semantics. This lifts the closed world assumption commonly made by lexicalized grammar supertaggers, greatly enhancing its generalization potential. This is evidenced both by its adequate accuracy over sparse word types and its ability to correctly construct complex types never seen during training, which, to the best of our knowledge, was as of yet unaccomplished.Comment: REPL4NLP 4, ACL 201

Fighting the COVID-19 Infodemic with a Holistic BERT Ensemble

This paper describes the TOKOFOU system, an ensemble model for misinformation detection tasks based on six different transformer-based pre-trained encoders, implemented in the context of the COVID-19 Infodemic Shared Task for English. We fine tune each model on each of the task's questions and aggregate their prediction scores using a majority voting approach. TOKOFOU obtains an overall F1 score of 89.7%, ranking first.Comment: 4 pages, NLP4IF 202

A Logic-Based Framework for Natural Language Inference in Dutch

We present a framework for deriving inference relations between Dutch sentence pairs. The proposed framework relies on logic-based reasoning to produce inspectable proofs leading up to inference labels; its judgements are therefore transparent and formally verifiable. At its core, the system is powered by two λ-calculi, used as syntactic and semantic theories, respectively. Sentences are first converted to syntactic proofs and terms of the linear λ-calculus using a choice of two parsers: an Alpino-based pipeline, and Neural Proof Nets. The syntactic terms are then converted to semantic terms of the simply typed λ-calculus, via a set of hand designed type- and term-level transformations. Pairs of semantic terms are then fed to an automated theorem prover for natural logic which reasons with them while using lexical relations found in the Open Dutch WordNet. We evaluate the reasoning pipeline on the recently created Dutch natural language inference dataset, and achieve promising results, remaining only within a 1.1−3.2% performance margin to strong neural baselines. To the best of our knowledge, the reasoning pipeline is the first logic-based system for Dutch

A Logic-Based Framework for Natural Language Inference in Dutch

We present a framework for deriving inference relations between Dutch sentence pairs. The proposed framework relies on logic-based reasoning to produce inspectable proofs leading up to inference labels; its judgements are therefore transparent and formally verifiable. At its core, the system is powered by two λ-calculi, used as syntactic and semantic theories, respectively. Sentences are first converted to syntactic proofs and terms of the linear λ-calculus using a choice of two parsers: an Alpino-based pipeline, and Neural Proof Nets. The syntactic terms are then converted to semantic terms of the simply typed λ-calculus, via a set of hand designed type- and term-level transformations. Pairs of semantic terms are then fed to an automated theorem prover for natural logic which reasons with them while using lexical relations found in the Open Dutch WordNet. We evaluate the reasoning pipeline on the recently created Dutch natural language inference dataset, and achieve promising results, remaining only within a 1.1−3.2% performance margin to strong neural baselines. To the best of our knowledge, the reasoning pipeline is the first logic-based system for Dutch

Discontinuous Constituency and BERT: A Case Study of Dutch

In this paper, we set out to quantify the syntactic capacity of BERT in the evaluation regime of non-context free patterns, as occurring in Dutch. We devise a test suite based on a mildly context-sensitive formalism, from which we derive grammars that capture the linguistic phenomena of control verb nesting and verb raising. The grammars, paired with a small lexicon, provide us with a large collection of naturalistic utterances, annotated with verb-subject pairings, that serve as the evaluation test bed for an attention-based span selection probe. Our results, backed by extensive analysis, suggest that the models investigated fail in the implicit acquisition of the dependencies examined

Neural Proof Nets

Linear logic and the linear {\lambda}-calculus have a long standing tradition in the study of natural language form and meaning. Among the proof calculi of linear logic, proof nets are of particular interest, offering an attractive geometric representation of derivations that is unburdened by the bureaucratic complications of conventional prooftheoretic formats. Building on recent advances in set-theoretic learning, we propose a neural variant of proof nets based on Sinkhorn networks, which allows us to translate parsing as the problem of extracting syntactic primitives and permuting them into alignment. Our methodology induces a batch-efficient, end-to-end differentiable architecture that actualizes a formally grounded yet highly efficient neuro-symbolic parser. We test our approach on {\AE}Thel, a dataset of type-logical derivations for written Dutch, where it manages to correctly transcribe raw text sentences into proofs and terms of the linear {\lambda}-calculus with an accuracy of as high as 70%.Comment: 14 pages, CoNLL202

OYXOY: A Modern NLP Test Suite for Modern Greek

This paper serves as a foundational step towards the development of a linguistically motivated and technically relevant evaluation suite for Greek NLP. We initiate this endeavor by introducing four expert-verified evaluation tasks, specifically targeted at natural language inference, word sense disambiguation (through example comparison or sense selection) and metaphor detection. More than language-adapted replicas of existing tasks, we contribute two innovations which will resonate with the broader resource and evaluation community. Firstly, our inference dataset is the first of its kind, marking not just \textit{one}, but rather \textit{all} possible inference labels, accounting for possible shifts due to e.g. ambiguity or polysemy. Secondly, we demonstrate a cost-efficient method to obtain datasets for under-resourced languages. Using ChatGPT as a language-neutral parser, we transform the Dictionary of Standard Modern Greek into a structured format, from which we derive the other three tasks through simple projections. Alongside each task, we conduct experiments using currently available state of the art machinery. Our experimental baselines affirm the challenging nature of our tasks and highlight the need for expedited progress in order for the Greek NLP ecosystem to keep pace with contemporary mainstream research

Dependency as Modality, Parsing as Permutation: A Neurosymbolic Perspective on Categorial Grammars

Type systems (in the form of categorial grammars) are the front runners in the quest for a formally elegant, computationally attractive and flexible theory of linguistic form and meaning. Words enact typed constants, and interact with one another via means of grammatical rules enacted by type inferences, composing larger phrases in the process. The end result is at the same time a parse, a proof and a program, bridging the seemingly disparate fields of linguistics, formal logics and computer science. The transition from form to meaning is traditionally handled via a series of homomorphisms that simplify nuances of the syntactic calculus to move towards a uniform semantic calculus. Alluring as this might be, it poses pragmatic considerations. For the setup to work on the semantic level, one has no choice but to start from the hardest part, namely natural language syntax. Phenomena like movement, word-order variation, discontinuities, etc. require careful treatment that needs to be both general enough to encompass the full range of grammatical utterances, yet strict enough to ward off ungrammatical derivations. This thesis takes an operational shortcut in targeting a ``deeper'' calculus of grammatical composition, engaging only minimally with surface syntax. Where previously functional functional syntactic types would be position-conscious, requiring their arguments in predetermined positions upon a binary tree, they are now agnostic to both tree structure and sequential order, alleviating the need for syntactic refinements. This simplification comes at the cost of a misalignment between provability and grammaticality: the laxer semantic calculus permits more proofs than linguistically desired. To circumvent this underspecification, the thesis takes a step away from the established norm, proposing the incorporation of unary type operators extending the function-argument axis with grammatical role labels. The new calculus produces mixed unary/n-ary trees, each unary tree denoting a dependency domain, and each n-ary tree denoting the phrases which together form that domain. Although still underspecified, these structures now subsume non-projective labeled dependency trees. More than that, they have their roots set firmly in type theory, allowing meaningful semantic interpretation. On more practical grounds and in order to investigate the formalism's adequacy, an extraction algorithm is employed to convert syntactic analyses of sentences (represented as dependency graphs) into proofs of the target logic. This gives rise to a large proofbank, a collection of sentences paired with tectogrammatic theorems and their corresponding programs, and an elaborate type lexicon, providing type assignments to one million lexical tokens within a linguistic context. The proofbank and the lexicon find use as training data in the design of a neurosymbolic proof search system, able to efficiently navigate the logic's theorem space. The system consists of three components. Component one is a supertagger responsible for assigning a type to each word — the tagger is formulated as a heterogeneous graph convolution kernel that boasts state-of-the-art accuracy. Rather than produce asignments in the form of conditional probabilities over a predefined vocabulary, it instead constructs types dynamicaly. As such, it is unconstrained by data sparsity, generalizing well to rare assignments and producing correct assignments for types never seen during training. Component two is a neural permutation module that exploits the linearity constraint of the logic in order to simplify proof search as optimal transport learning, associating resources (conditional validities) to the processes that require them (conditions). This allows for a parallel and easily optimizable implementation, unobstructed by the structure manipulation found in conventional parsers. Component three is the type system itself, responsible for navigating the produced structures and asserting their well-formedness. Results suggest performance superior to established baselines across categorial formalisms, despite the ambiguity inherent to the logic